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management  system  (CLAM).  A  CLAM  system  monitors  operators’  psychophysiology  for  signs  of  inattention  and  then  triggers  a 
countermeasure  to  rouse  operators  and  help  them  sustain  vigilance  and  good  task  performance.  There  are  many  requirements  for  bringing 
this  concept  to  fruition,  including  minimal  or  no  contact  psychophysiological  measures,  accurate  and  precise  prediction  of  attention  level 
and  task  performance,  and  effective  interface  modifications.  Over  the  course  of  this  three  year  project,  we  have  investigated  1 )  effective 
combinations  of  psychophysiological  measures  of  inattention,  2)  effective  countermeasures,  and  3)  a  complete  closed-loop  system  for 
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journal  article  is  under  review. 
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Vigilance  tasks,  from  driving  to  surveillance  to  security  remain  important  and  frequent 
tasks  for  the  US  Army.  Yet  the  difficulty  users  have  sustaining  vigilance  is  well  known. 
Augmented  cognition  offers  new  methods  for  supporting  sustained  vigilance  via  a  closed-loop 
attention  management  system  (CLAM).  A  CLAM  system  monitors  operators’  psychophysiology 
for  signs  of  inattention  and  then  triggers  a  countermeasure  to  rouse  operators  and  help  them 
sustain  vigilance  and  good  task  performance.  There  are  many  requirements  for  bringing  this 
concept  to  fruition,  including  minimal  or  no  contact  psychophysiological  measures  that  are 
minimally  invasive  or  constraining,  accurate  and  precise  prediction  of  attention  level  and  task 
performance,  and  effective  interface  modifications.  Over  the  course  of  this  three  year  project,  we 
have  investigated  1)  effective  combinations  of  psychophysiological  measures  of  inattention,  2) 
effective  countermeasures  to  rouse  and  re-engage  participants  once  inattention  is  detected,  and  3) 
a  complete  closed-loop  system  for  monitoring  and  sustaining  attention  and  task  performance. 

The  project  has  produced  three  book  chapters  and  one  conference  paper.  A  journal  article  is 
under  review.  Below  are  abstracts  describing  the  work  from  each  year  of  the  project. 


ilb  1  A  Real-Time  Closed-Loop  System  for  Predicting  and 

%  X Counteracting  Lapses  in  Attention  |  m  | 

Mark  St.  John,  Pacific  Science  &  Engineering  Group 

Objective 

•  Augment  sustained  attention  by  developing 
closed-loop  systems  to  monitor  attention 
and  activate  countermeasures 

•  Many  Army  tasks  require  operators  to 
sustain  attention  over  long  periods  of  time 

•  Lapses  may  be  detrimental  to  performance 
and  potentially  dangerous 

•  Inform  theories  of  attention  and  its 
modulation  via  close-loop  systems 

•  Wireless  EEG 
head  gear 

•  Noncontact  eye 
and  head 
tracking 

•  Graph  of  eye 
lid  opening 
(left) 

•  Graph  of  head 
pitch  (right) 

•  Graph  ofEEG 
(bottom) 

Approach 

•  Year  1 :  Develop  vigilance  tasks  with  clear 
Army  relevance  —  UAV  video  surveillance 

•  Integrate  real-time  psychophysiological 
measures  of  attention 

•  Year  2:  Evaluate  alternative  secondary  task 
countermeasures 

•  Year  3:  Develop  a  closed-loop  system  to 
monitor  attention  and  intervene  to  maintain 
high  vigilance  task  performance 

Status 

•  Developed  closed-loop  system  to 

•  Detect  inattention  using  EEG,  eye,  and  head 
tracking 

•  Activate  countermeasures 

•  “Gold  standard”  comparison 

•  Closed-loop  with  countermeasures  tailored 
to  inattention  vs.  open-loop  randomly 
activated  countermeasures 

•  Closed-loop  vigilance  performance  17% 
better 

Year  1.  In  the  present  study,  we  report  a  novel  combination  of  no  contact  head  and  eye 
measures  combined  with  a  wireless  EEG  measure  to  predict  performance  on  a  sustained 
vigilance  task.  Each  measure  was  computed  for  10  five  minute  blocks  over  the  course  of  the 
experiment.  The  four  most  predictive  variables  were  eye  opening  (vertical  distance  between 
eyelids),  head  pitch  variability  (amount  of  nodding),  high  vigilance  (a  measure  derived  from  a 
spectral  analysis  of  the  EEG  signal),  and  the  sum  of  the  high  and  low  vigilance  indices  (Berka  et 
al.,  2004).  Together,  the  measures  predicted  42%  of  the  variance  in  the  miss  rate  (39%  of  the 
variance  in  A’).  No  individual  variable  accounted  for  more  than  13%  of  the  variance.  Separate 
multiple  regressions  using  the  same  four  variables  were  computed  for  each  participant.  The 
percentage  of  variance  in  the  miss  rate  accounted  for  varied  from  32%  to  91%.  These  findings 
suggest  that  these  minimally  invasive  measures,  used  in  combination,  may  be  sensitive  enough 
and  acceptable  enough  for  use  in  closed-loop  attention  management  systems  in  a  variety  of 
operational  contexts  that  require  optimal  continuous  performance. 

Year  2.  The  present  study  presents  and  tests  a  novel  hybrid  model  of  the  underlying 
mechanisms  responsible  for  the  vigilance  decrement  that  combines  aspects  of  both  arousal  and 
resource  theories.  From  the  new  Arousal-Resource  model,  we  derived  and  evaluated  three 
alternative  interventions.  The  interventions  took  the  form  of  secondary  tasks  designed  to  draw 
upon  resources  separate  from  those  required  by  the  primarily  visual  vigilance  task:  an  auditory 
alarm  “ring  tone”  that  exercised  sensory  perception  only  and  two  auditory  cognitive  tasks  that 
additionally  exercised  working  memory  by  requiring  participants  to  mentally  reorder  strings  of  3 
or  4  spoken  digits.  We  hypothesized  that  the  cognitive  digit  task  interventions  would  arouse 
participants,  replenish  depleted  resources,  and  re-engage  them  in  the  vigilance  task.  We  further 
hypothesized  that  the  cognitive  tasks  would  prove  more  effective  than  the  simple  alarm  because 
they  are  more  demanding  and  engaging.  Participants  performed  a  standard  45  minute  laboratory 
vigilance  task  twice,  once  in  a  control  condition  without  any  intervention  and  once  with  one  of 
the  three  different  interventions.  In  the  intervention  conditions,  participants  received  the 
intervention  whenever  they  missed  a  target.  This  method  for  intervention  served  as  a  convenient 
proxy  for  a  closed- loop  system  in  which  participants  would  receive  interventions  whenever  low 
attention  was  detected  by  psychophysiological  measures,  prior  to  an  actual  miss.  All  three 
interventions  significantly  reduced  misses  by  approximately  30%.  Participants  who  showed 
greater  vigilance  decrements  in  the  baseline  condition  showed  more  improvement  from  all 
interventions.  Therefore,  more  vulnerable  participants  benefited  most.  The  cognitive 
interventions  performed  as  well  as,  but  no  better  than,  the  simple  alarm.  The  cognitive  tasks  also 
interfered  with  target  detection  performance  on  occasions  when  the  interventions  were  active. 
However,  the  alarm  was  rated  subjectively  as  more  frustrating  and  less  appropriate  than  either 
cognitive  intervention.  Because  the  added  task  demands  of  the  cognitive  interventions  improved 
vigilance  task  performance  rather  than  harming  it,  the  Arousal-Resource  model  is  supported  by 
the  results.  Secondary  tasks  may  be  used  to  arouse  and  re-engage  participants,  and  participants 
appear  to  prefer  these  interventions  to  alarms.  However,  care  should  be  taken  to  minimize 
interference  between  the  primary  and  secondary  tasks. 

Year  3.  Here,  we  report  an  evaluation  of  a  complete  closed-loop  system  composed  of  a 
combination  of  eye,  head,  and  EEG  measures  and  a  novel  countermeasure  composed  of  a 
cognitively  demanding  secondary  task.  In  order  to  evaluate  the  CLAM  system,  the  secondary 
task  was  triggered  either  when  inattention  was  detected  (CLAM)  or  at  random  intervals 
throughout  a  40  minute  vigilance  task.  While  participants  in  both  conditions  demonstrated  a 


vigilance  decrement,  as  measured  by  an  increase  in  misses  over  the  course  of  the  session,  the 
CLAM  condition  produced  17%  fewer  misses  overall  than  the  random  condition.  These  results 
indicate  successful  real-time  detection  of  inattention  and  an  effective  countermeasure  for  rousing 
participants  and  sustaining  vigilance  and  task  performance.  The  results  inform  our  understanding 
of  how  human  vigilance  operates  and  the  technology  for  its  detection  and  manipulation. 
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